Skip to content

10 create end to end trade management logic#14

Merged
mccaffers merged 25 commits into
mainfrom
10-create-end-to-end-trade-management-logic
May 2, 2026
Merged

10 create end to end trade management logic#14
mccaffers merged 25 commits into
mainfrom
10-create-end-to-end-trade-management-logic

Conversation

@mccaffers

Copy link
Copy Markdown
Owner

No description provided.

@mccaffers mccaffers self-assigned this Apr 12, 2026
Copilot AI review requested due to automatic review settings April 12, 2026 07:57
@mccaffers mccaffers linked an issue Apr 12, 2026 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial end-to-end “run” path for pulling tick data from QuestDB, parsing a Base64 strategy config, and iterating through streamed ticks (with supporting model/API updates).

Changes:

  • Introduces Base64→JSON configuration parsing and a new Operations::run() tick-processing entry point.
  • Reworks DB access to build symbol-union queries and parse results into an updated PriceData model that includes ask/bid/timestamp/symbol.
  • Updates CMake/OpenMP linkage and scripts/docs to support running the new flow.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 20 comments.

Show a summary per file
File Description
source/utilities/jsonParser.cpp New Base64 JSON configuration parser used by CLI.
source/sqlManager.cpp Builds multi-symbol query and calls DB streaming API.
source/operations.cpp Adds tick loop/printing “operations” runner.
source/main.cpp Wires config parsing + tick streaming + operations run into CLI.
source/databaseConnection.cpp Adds faster timestamp parsing and new streamQuery() result parsing.
source/CMakeLists.txt Adds a duplicate CMake build file for the source/ subtree.
scripts/run.sh Adds build-failure guard and execution timing.
scripts/build_dep.sh Adds a helper script to build libpqxx dependency.
include/sqlManager.hpp Updates SqlManager API to streamPriceData/symbol list.
include/operations.hpp Declares new Operations runner.
include/models/priceData.hpp Updates tick model fields and constructors.
include/databaseConnection.hpp Adds streamQuery() declaration (keeps executeQuery() declaration).
documents/questdb.md Adds a short note on starting QuestDB (macOS).
CMakeLists.txt Adds OpenMP linking for the main build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread source/sqlManager.cpp
Comment thread source/sqlManager.cpp Outdated
Comment on lines +15 to +21
std::string query;
for (size_t i = 0; i < symbols.size(); ++i) {
if (i > 0) {
query += " UNION ALL ";
}
query += "SELECT '" + symbols[i] + "' as symbol, * FROM '" + symbols[i] + "' WHERE timestamp >= dateadd('M', -" + std::to_string(LAST_MONTHS) + ", now())";
}

Copilot AI Apr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated SQL uses single quotes around the table name (FROM 'EURUSD'), which in SQL denotes a string literal, not an identifier. This will produce invalid SQL (or query the wrong thing). Use an unquoted identifier or proper identifier quoting (typically double quotes), and ensure symbols are validated/escaped to avoid SQL injection via symbols[i].

Copilot uses AI. Check for mistakes.
Comment thread source/sqlManager.cpp Outdated
Comment on lines +10 to +13
std::string SqlManager::getBaseQuery(const std::vector<std::string>& symbols, int LAST_MONTHS) {
if (symbols.empty()) {
return "";
}

Copilot AI Apr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getBaseQuery returns an empty string when symbols is empty, but streamPriceData will still execute the query string it gets back. Returning an empty query can lead to runtime SQL errors; consider returning an empty result early in streamPriceData (or throwing) when symbols is empty.

Copilot uses AI. Check for mistakes.
Comment thread source/operations.cpp
Comment thread source/operations.cpp Outdated
Comment on lines +31 to +37
struct tm tm = {};
if (localtime_r(&time_t, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;
}
char buffer[20];
std::strftime(buffer, sizeof(buffer), "%Y-%m-%d %H:%M:%S", &tm);
auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(tick.timestamp.time_since_epoch()) % 1000;

Copilot AI Apr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If localtime_r fails, the code logs an error but then continues to call std::strftime using an uninitialized tm value. This can print garbage timestamps. Consider continue; (or otherwise handling the error) after the failure so the formatting code is skipped.

Copilot uses AI. Check for mistakes.
Comment thread source/CMakeLists.txt Outdated
Comment thread source/CMakeLists.txt
Comment on lines +54 to +65
if(APPLE)
set(OpenMP_C_FLAGS "-Xclang -fopenmp")
set(OpenMP_CXX_FLAGS "-Xclang -fopenmp")
set(OpenMP_C_LIB_NAMES "omp")
set(OpenMP_CXX_LIB_NAMES "omp")
set(OpenMP_omp_LIBRARY /opt/homebrew/opt/libomp/lib/libomp.dylib)
find_package(OpenMP REQUIRED)
target_include_directories(BacktestingEngineLib PRIVATE /opt/homebrew/opt/libomp/include)
endif()

target_link_libraries(BacktestingEngineLib PUBLIC pqxx OpenMP::OpenMP_CXX)

Copilot AI Apr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This CMake file links OpenMP::OpenMP_CXX unconditionally, but find_package(OpenMP REQUIRED) is only called inside if(APPLE). On non-Apple platforms this will fail because the imported target won’t exist. Consider calling find_package(OpenMP REQUIRED) for all platforms (keeping only the Apple-specific hints in the if(APPLE) block) or conditionally linking OpenMP when found.

Copilot uses AI. Check for mistakes.
Comment thread scripts/build_dep.sh
Comment on lines +1 to +3
cd ./external/libpqxx

mkdir -p build

Copilot AI Apr 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script is missing a shebang (e.g., #!/bin/bash or #!/usr/bin/env bash). As-is, running it directly (instead of sh scripts/build_dep.sh) may fail depending on permissions/default shell. Add a shebang and consider set -euo pipefail so dependency builds fail fast.

Copilot uses AI. Check for mistakes.
Comment thread source/databaseConnection.cpp Outdated
Comment thread scripts/run.sh Outdated
mccaffers and others added 8 commits April 12, 2026 09:37
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…com:mccaffers/backtesting-engine-cpp into 10-create-end-to-end-trade-management-logic

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 15 comments.

Comments suppressed due to low confidence (2)

source/utilities/jsonParser.cpp:19

  • This function writes the decoded JSON and RUN_ID to stdout unconditionally. That can leak potentially sensitive configuration data and also makes the parser noisy for library callers/tests; consider removing these prints or gating them behind an explicit debug/logging facility.
    source/utilities/jsonParser.cpp:31
  • This function now returns a Configuration, but j.get<trading_definitions::Configuration>() will throw if parsing failed (the earlier parse error is only logged). Consider making the error handling explicit (rethrow/return an error type) so callers don’t proceed with an invalid/partial configuration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread source/operations.cpp Outdated
Comment on lines +44 to +45
auto tradeManager = TradeManager::getInstance();

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tradeManager is retrieved but never used (all example calls are commented). This will trigger unused-variable warnings and adds dead code. Either remove it for now or implement the intended trade-management flow so Operations::run has a concrete effect.

Copilot uses AI. Check for mistakes.
Comment thread source/operations.cpp Outdated
Comment on lines +29 to +41
// (void)tick;

// print first tick
auto time_t = std::chrono::system_clock::to_time_t(tick.timestamp);
struct tm tm = {};
if (localtime_r(&time_t, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;
}
char buffer[20];
std::strftime(buffer, sizeof(buffer), "%Y-%m-%d %H:%M:%S", &tm);
auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(tick.timestamp.time_since_epoch()) % 1000;
// printf("symbol=%s, ask=%.4f, bid=%.4f timestamp=%s.%03lld\n", tick.symbol.c_str(), tick.ask, tick.bid, buffer,
// static_cast<long long>(ms.count()));

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inside the tick loop, expensive timestamp formatting work is done for every tick even though the output is commented out. This will significantly slow large backtests; remove this work or gate it behind a debug flag / only sample a small number of ticks.

Suggested change
// (void)tick;
// print first tick
auto time_t = std::chrono::system_clock::to_time_t(tick.timestamp);
struct tm tm = {};
if (localtime_r(&time_t, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;
}
char buffer[20];
std::strftime(buffer, sizeof(buffer), "%Y-%m-%d %H:%M:%S", &tm);
auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(tick.timestamp.time_since_epoch()) % 1000;
// printf("symbol=%s, ask=%.4f, bid=%.4f timestamp=%s.%03lld\n", tick.symbol.c_str(), tick.ask, tick.bid, buffer,
// static_cast<long long>(ms.count()));
(void)tick;

Copilot uses AI. Check for mistakes.
Comment thread source/operations.cpp Outdated
Comment on lines +32 to +35
auto time_t = std::chrono::system_clock::to_time_t(tick.timestamp);
struct tm tm = {};
if (localtime_r(&time_t, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If localtime_r fails, the code logs an error but still calls strftime on tm, which can produce misleading output; consider continue on failure. Also rename time_t to avoid shadowing the time_t type name.

Suggested change
auto time_t = std::chrono::system_clock::to_time_t(tick.timestamp);
struct tm tm = {};
if (localtime_r(&time_t, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;
auto tick_time = std::chrono::system_clock::to_time_t(tick.timestamp);
struct tm tm = {};
if (localtime_r(&tick_time, &tm) == nullptr) {
std::cerr << "Error: failed to convert timestamp" << std::endl;
continue;

Copilot uses AI. Check for mistakes.
Comment thread source/CMakeLists.txt
Comment on lines +24 to +28
# Disable warningsfor external libraries
set(PREV_CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS})
if(CMAKE_CXX_COMPILER_ID MATCHES "Clang|GNU")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -w")
elseif(MSVC)

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This CMakeLists globally appends -w//w, which disables warnings for all targets (including your own code) and can mask real issues. Prefer limiting warning suppression to third-party targets only (e.g., via per-target compile options or SYSTEM includes).

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/sonarcloud.yml Outdated
Comment on lines 94 to 97
- name: Build C++ Libraries
run: >
bash ./.github/workflows/build.sh
sh ./scripts/build.sh
- name: Install Python 3.12 for gcovr

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Ubuntu Sonar job runs sh ./scripts/build.sh, but that script relies on macOS tooling (Homebrew + xcrun) and is expected to fail on Ubuntu. Replace this with Linux-compatible build steps (apt deps + cmake) or make scripts/build.sh OS-aware.

Copilot uses AI. Check for mistakes.
Comment thread source/CMakeLists.txt
Comment on lines +49 to +50
# Collect all .cpp files in the src directory
file(GLOB_RECURSE SOURCES "source/*.cpp")

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

file(GLOB_RECURSE SOURCES "source/*.cpp") inside source/CMakeLists.txt will look for source/source/*.cpp relative to this CMakeLists, which is likely not intended and can produce an empty library. Use paths relative to the correct root (e.g., "*.cpp" if this file is the project root) or switch to explicit target sources to avoid fragile globs.

Suggested change
# Collect all .cpp files in the src directory
file(GLOB_RECURSE SOURCES "source/*.cpp")
# Collect all .cpp files in the current source directory
file(GLOB_RECURSE SOURCES "*.cpp")

Copilot uses AI. Check for mistakes.
Comment thread source/sqlManager.cpp Outdated
Comment on lines +21 to +22
query += "SELECT '" + symbols[i] + "' as symbol, * FROM '" + symbols[i] + "' WHERE timestamp >= dateadd('M', -" + std::to_string(LAST_MONTHS) + ", now())";
}

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

symbols[i] is concatenated directly into the SQL string (including the FROM clause), so a malicious or malformed symbol can inject SQL or break the query. Validate symbols against an allowlist and/or implement safe identifier escaping rather than raw string concatenation.

Copilot uses AI. Check for mistakes.
Comment on lines +22 to +26
// Cache timegm per date — tick data is time-ordered so date changes rarely
static char cachedDate[11] = {};
static time_t cachedEpoch = 0;
if (std::memcmp(ts, cachedDate, 10) != 0) {
std::memcpy(cachedDate, ts, 10);

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastParseTimestamp uses shared mutable function-local static cache state. If streamQuery is called concurrently, this is a data race (UB). Consider making the cache thread_local or moving it into a caller-owned object.

Copilot uses AI. Check for mistakes.
Comment thread scripts/build.sh
Comment on lines +25 to 30
cmake .. \
-DCMAKE_CXX_STANDARD=20 \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path) \
-DSKIP_BUILD_TEST=ON

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script always passes -DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path) to CMake; xcrun won’t exist on Linux, and this script is invoked from the Ubuntu Sonar job. Guard the xcrun call or only pass CMAKE_OSX_SYSROOT on Apple platforms.

Suggested change
cmake .. \
-DCMAKE_CXX_STANDARD=20 \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path) \
-DSKIP_BUILD_TEST=ON
CMAKE_ARGS=".. \
-DCMAKE_CXX_STANDARD=20 \
-DCMAKE_BUILD_TYPE=Release \
-DSKIP_BUILD_TEST=ON"
if [ "$(uname)" = "Darwin" ] && command -v xcrun &>/dev/null; then
CMAKE_ARGS="$CMAKE_ARGS -DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path)"
fi
cmake $CMAKE_ARGS

Copilot uses AI. Check for mistakes.
Comment thread source/main.cpp Outdated
for (std::string token; std::getline(ss, token, ',');) {
symbols.push_back(token);
}
std::vector<PriceData> ticks = SqlManager::streamPriceData(db, symbols, 3);

Copilot AI Apr 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LAST_MONTHS is provided by the decoded configuration but this call hard-codes 3, so the CLI config is ignored. Use config.LAST_MONTHS (with validation) so backtests match the provided config.

Suggested change
std::vector<PriceData> ticks = SqlManager::streamPriceData(db, symbols, 3);
// Use the configured lookback window for backtesting and validate it
int lastMonths = config.LAST_MONTHS;
if (lastMonths <= 0) {
std::cerr << "Invalid LAST_MONTHS value in configuration: " << lastMonths
<< ". Expected a positive integer." << std::endl;
return 1;
}
std::vector<PriceData> ticks = SqlManager::streamPriceData(db, symbols, lastMonths);

Copilot uses AI. Check for mistakes.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
18.2% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@mccaffers mccaffers merged commit 617c263 into main May 2, 2026
1 check passed
mccaffers added a commit that referenced this pull request May 26, 2026
feat: add QuestDB tick streaming with multi-symbol support and refactored trade flow

* Integrate QuestDB tick data streaming with multi-symbol loop support
* Extract strategy logic for querying QuestDB into dedicated module
* Refactor tick flow, operations flow, and trade management operations
* Fix run script shell scoping issue (build script running in child process)
* Update and stabilise GitHub Actions workflow (macOS target, build & test optimisation)
* Update CMakeLists.txt for updated source structure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create end to end trade management logic

3 participants